1. Field of Invention
The field of the currently claimed embodiments of this invention relates to robotic systems and more particularly to semi-automatic, interactive robotic systems and simulated semi-automatic, interactive robotic systems.
2. Discussion of Related Art
Robotic systems can be useful for the performance of repetitive, complex, intricate and/or dangerous tasks. As applied to surgery, for example, complex procedures represent a high workload for the surgeon. In addition, the recent introduction of robots into the surgery room has led to the need for new techniques to train and evaluate surgeons. For this reason, surgical gesture modeling has attracted significant attention in recent years, and several methods, usually using Hidden Markov Models or variations thereof, have been proposed for off-line skill modeling and classification (J. Rosen, J. Brown, L. Chang, M. Sinanan, and B. Hannaford, “Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete markov model,” IEEE Trans, on Biomedical Engineering, vol. 53, no. 3, pp. 399-413, 2006; H. C. Lin, I. Shafran, D. Yuh, and G. D. Hager, “Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions,” Computer Aided Surgery, vol. 11, no. 5, pp. 220-230, 2006; B. Varadarajan, C. E. Reiley, H. Lin, S. Khudanpur, and G. D. Hager, “Data-derived models for segmentation with application to surgical assessment and training,” in MICCAI (1), 2009, pp. 426-434).
With the development of dexterous robots, different groups have proposed techniques to automate specific surgical tasks. An example of a task that has been addressed is knot tying. In (H. G. Mayer, F. J. Gomez, D. Wierstra, I. Nagy, A. Knoll, and J. Schmidhuber, “A system for robotic heart surgery that learns to tie knots using recurrent neural networks,” in IROS, 2006, pp. 543-548), recurrent neural networks are used to learn a loop trajectory from demonstration. In (J. van den Berg, S. Miller, D. Duckworth, H. Hu, A. Wan, X.-Y. Fu, K. Goldberg, and P. Abbeel, “Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations,” in ICRA, 2010, pp. 2074-2081), robot dynamics are learned to replay the trajectory at an increased speed. The needle insertion task has also attracted much attention: a geometric task model is designed in (F. Nageotte, P. Zanne, C. Doignon, and M. de Mathelin, “Stitching planning in laparoscopic surgery: Towards robot-assisted suturing,” Robotic Res., vol. 28, no. 10, pp. 1303-1321, 2009) to compute the path-planning of the needle insertion. In (C. Staub, T. Osa, A. Knoll, and R. Bauernschmitt, “Automation of tissue piercing using circular needles and vision guidance for computer aided laparoscopic surgery,” in ICRA, 2010, pp. 4585-4590), a circular motion is automated for needle insertion after the surgeon has marked the insertion point with a laser-pointer. There remains, however, a need for automation methods that either deal with the environment such as tissues and suture threads or that provides collaboration with the operator.
A natural way to allow for collaboration between a robot and the operator is to change the interaction mode based on the current context. This has been demonstrated on a curve following task in microsurgery by using virtual fixtures to impose path constraints on the manipulator (D. Kragic and G. Hager, “Task modeling and specification for modular sensory based human-machine cooperative systems,” in IROS, 2003, pp. 3192-3197).
Context modeling for real-time recognition of the current surgical task has been addressed in                B. P. L. Lo, A. Darzi, and G.-Z. Yang, “Episode classification for the analysis of tissue/instrument interaction with multiple visual cues,” in International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2003, pp. 230-237.        K. Yoshimitsu, F. Miyawaki, T. Sadahiro, K. Ohnuma, Y. Fukui, D. Hashimoto, and K. Masamune, “Development and evaluation of the second version of scrub nurse robot (snr) for endoscopic and laparoscopic surgery,” in IROS, 2007, pp. 2288-2294.        S. Speidel, G. Sudra, J. Senemaud, M. Drentschew, B. P. Mller-Stich, C. Gutt, and R. Dillmann, “Recognition of risk situations based on endoscopic instrument tracking and knowledge based situation modeling,” in Med. Imaging. SPIE, 2008.        N. Padoy, D. Mateus, D. Weinland, M.-O. Berger, and N. Navab, “Workflow monitoring based on 3d motion features,” in Proceedings of the International Conference on Computer Vision Workshops, IEEE Workshop on Video-oriented Object and Event Classification, 2009.typically using automata or Hidden Markov Models. However, these approaches do not allow for human-machine interactions to perform the operation. There thus remains a need for improved human-machine collaborative robotic systems.        